Solving single molecules: filtering noisy discrete data made of photons and other type of observables

In numerous systems in biophysics and related fields, scientists measure (with very smart methods) individual molecules (e.g. biopolymers (proteins, DNA, RNA, etc), nano – crystals, ion channels), aiming at finding a model from the data. But the noise is not solved accurately in not so few cases and this may lead to misleading models. Here, we solve the noise. We consider two state photon trajectories from any on off kinetic scheme (KS): the process emitting photons with a rate γon when it is in the on state, and emitting with a rate γoff when it is in the off state. We develop a filter that removes the noise resulting in clean data also in cases where binning fails. The filter is a numerical algorithm with various new statistical treatments. It is based on a new general likelihood function developed here, with observable dependent form. The filter can solve the noise, in the detectable region of the rate space: that is, we also find a region where the data is “too” noisy. Consistency tests will find the region’s type from the data. If the data is ruled “too noisy”, binning obviously fails, and one should apply simpler methods on the raw data and realizing that the extracted information is partial. We show that not applying the filter while cleaning results in erroneous rates. This filter (with minor adjustments) can solve the noise in any discrete state trajectories, yet extensions are needed in “tackling” the noise from other data, e.g. continuous data and FRET data. The filter developed here is complementary with our previous projects in this field, where we have solved clean two state data with the development of reduced dimensions forms (RDFs): unique models that are canonical forms of two state data, and with the development of a statistical and numerical toolbox that builds a RDF from finite, clean, two state data. Thus, only